Development and management of collective network and cloud computing infrastructures

Pla de Doctorat industrial de la Generalitat de CatalunyaIn the search and development of more participatory models for infrastructure development and management, in this dissertation, we investigate models for the financing, deployment, and operation of network and cloud computing infrastructures. Our main concern is to overcome the inherent exclusion in participation in the processes of development and management and in the right of use in the current dominant models. Our work starts by studying in detail the model of Guifi.net, a successful bottom-up initiative for building network infrastructure, generally referred to as a community networks. We pay special attention to its governance system and economic organisation because we argue that these are the key components of the success of this initiative. Then, we generalise our findings for any community network, aiming at becoming sustainable and scalable, and we explore the suitability of the Guifi.net model to the cloud computing infrastructure. As a result of both, we coin the attribute extensible to refer to infrastructure that is relatively easy to expand and maintain in contrast to those naturally limited or hard to expand, such as natural resources or highly complex or advanced artificial systems. We conclude proposing a generic model which, in our opinion, is suitable, at least, for managing extensible infrastructure. The Guifi.net model is deeply rooted in the commons; thus, the research in this field, in general, and Elinor Ostrom’s work, in particular, have left a profound imprint in our work. Our results show that the \guifinet model meets almost entirely the principles of long-enduring commons identified by E. Ostrom. This work has been developed as an industrial doctorate. As such, it combines academic research with elements of practice and pursues an effective knowledge transfer between academia and the private sector. Given that the private sector’s partner is a not-for-profit organisation, the effort to create social value has prevailed over the ambition to advance the development of a specific industrial product or particular technology.En la recerca i desenvolupament de models més participatius per al desenvolupament i gestió d'infraestructura, en aquesta tesi investiguem sobre models per al finançament, desplegament i operació d'infraestructures de xarxa i de computació al núvol. La nostra preocupació principal és fer front a l’exclusió inherent dels models dominants actualment pel que fa a la participació en els processos de desenvolupament i gestió i, també, als drets d’us. El nostre treball comença amb un estudi detallat del model de Guifi.net, un cas d'èxit d'iniciativa ciutadana en la construcció d'infraestructura de xarxa, iniciatives que es coneixen com a xarxes comunitàries. En fer-ho, parem una atenció especial al sistema de governança i a l’organització econòmica perquè pensem que són els dos elements claus de l'èxit d'aquesta iniciativa. Tot seguit passem a analitzar d'altres xarxes comunitàries per abundar en la comprensió dels factors determinants per a la seva sostenibilitat i escalabilitat. Després ampliem el nostre estudi analitzant la capacitat i el comportament del model de Guifi.net en el camp de les infraestructures de computació al núvol. A resultes d'aquests estudis, proposem l'atribut extensible per a descriure aquelles infraestructures que són relativament fàcil d'ampliar i gestionar, en contraposició a les que o bé estan limitades de forma natural o be són difícils d'ampliar, com ara els recursos naturals o els sistemes artificials avançats o complexos. Finalitzem aquest treball fent una proposta de model genèric que pensem que és d'aplicabilitat, com a mínim, a tot tipus d'infraestructura extensible. El model de Guifi.net està fortament vinculat als bens comuns. És per això que la recerca en aquest àmbit, en general, i els treballs de Elinor Ostrom en particular, han deixat una forta empremta en el nostre treball. Els resultats que hem obtingut mostren que el model Guifi.net s'ajusta molt bé als principis que segons Ostrom han de complir els béns comuns per ser sostenibles. Aquest treball s'ha desenvolupat com a doctorat industrial. Com a tal, combina la investigació acadèmica amb elements de practica i persegueix una transferència efectiva de coneixement entre l'àmbit acadèmic i el sector privat. Ates que el soci del sector privat és una organització sense ànim de lucre, l’esforç per crear valor social ha prevalgut en l’ambició d’avançar en el desenvolupament d'un producte industrial específic o d'una tecnologia particularPostprint (published version

Development and management of collective network and cloud computing infrastructures

In the search and development of more participatory models for infrastructure development and management, in this dissertation, we investigate models for the financing, deployment, and operation of network and cloud computing infrastructures. Our main concern is to overcome the inherent exclusion in participation in the processes of development and management and in the right of use in the current dominant models. Our work starts by studying in detail the model of Guifi.net, a successful bottom-up initiative for building network infrastructure, generally referred to as a community networks. We pay special attention to its governance system and economic organisation because we argue that these are the key components of the success of this initiative. Then, we generalise our findings for any community network, aiming at becoming sustainable and scalable, and we explore the suitability of the Guifi.net model to the cloud computing infrastructure. As a result of both, we coin the attribute extensible to refer to infrastructure that is relatively easy to expand and maintain in contrast to those naturally limited or hard to expand, such as natural resources or highly complex or advanced artificial systems. We conclude proposing a generic model which, in our opinion, is suitable, at least, for managing extensible infrastructure. The Guifi.net model is deeply rooted in the commons; thus, the research in this field, in general, and Elinor Ostrom’s work, in particular, have left a profound imprint in our work. Our results show that the \guifinet model meets almost entirely the principles of long-enduring commons identified by E. Ostrom. This work has been developed as an industrial doctorate. As such, it combines academic research with elements of practice and pursues an effective knowledge transfer between academia and the private sector. Given that the private sector’s partner is a not-for-profit organisation, the effort to create social value has prevailed over the ambition to advance the development of a specific industrial product or particular technology.En la recerca i desenvolupament de models més participatius per al desenvolupament i gestió d'infraestructura, en aquesta tesi investiguem sobre models per al finançament, desplegament i operació d'infraestructures de xarxa i de computació al núvol. La nostra preocupació principal és fer front a l’exclusió inherent dels models dominants actualment pel que fa a la participació en els processos de desenvolupament i gestió i, també, als drets d’us. El nostre treball comença amb un estudi detallat del model de Guifi.net, un cas d'èxit d'iniciativa ciutadana en la construcció d'infraestructura de xarxa, iniciatives que es coneixen com a xarxes comunitàries. En fer-ho, parem una atenció especial al sistema de governança i a l’organització econòmica perquè pensem que són els dos elements claus de l'èxit d'aquesta iniciativa. Tot seguit passem a analitzar d'altres xarxes comunitàries per abundar en la comprensió dels factors determinants per a la seva sostenibilitat i escalabilitat. Després ampliem el nostre estudi analitzant la capacitat i el comportament del model de Guifi.net en el camp de les infraestructures de computació al núvol. A resultes d'aquests estudis, proposem l'atribut extensible per a descriure aquelles infraestructures que són relativament fàcil d'ampliar i gestionar, en contraposició a les que o bé estan limitades de forma natural o be són difícils d'ampliar, com ara els recursos naturals o els sistemes artificials avançats o complexos. Finalitzem aquest treball fent una proposta de model genèric que pensem que és d'aplicabilitat, com a mínim, a tot tipus d'infraestructura extensible. El model de Guifi.net està fortament vinculat als bens comuns. És per això que la recerca en aquest àmbit, en general, i els treballs de Elinor Ostrom en particular, han deixat una forta empremta en el nostre treball. Els resultats que hem obtingut mostren que el model Guifi.net s'ajusta molt bé als principis que segons Ostrom han de complir els béns comuns per ser sostenibles. Aquest treball s'ha desenvolupat com a doctorat industrial. Com a tal, combina la investigació acadèmica amb elements de practica i persegueix una transferència efectiva de coneixement entre l'àmbit acadèmic i el sector privat. Ates que el soci del sector privat és una organització sense ànim de lucre, l’esforç per crear valor social ha prevalgut en l’ambició d’avançar en el desenvolupament d'un producte industrial específic o d'una tecnologia particula

Cloudy in guifi.net: Establishing and sustaining a community cloud as open commons

Commons are natural or human-made resources that are managed cooperatively. The guifi.net community network is a successful example of a digital infrastructure, a computer network, managed as an open commons. Inspired by the guifi.net case and its commons governance model, we claim that a computing cloud, another digital infrastructure, can also be managed as an open commons if the appropriate tools are put in place. In this paper, we explore the feasibility and sustainability of community clouds as open commons: open user-driven clouds formed by community-managed computing resources. We propose organising the infrastructure as a service (IaaS) and platform as a service (PaaS) cloud service layers as common-pool resources (CPR) for enabling a sustainable cloud service provision. On this basis, we have outlined a governance framework for community clouds, and we have developed Cloudy, a cloud software stack that comprises a set of tools and components to build and operate community cloud services. Cloudy is tailored to the needs of the guifi.net community network, but it can be adopted by other communities. We have validated the feasibility of community clouds in a deployment in guifi.net of some 60 devices running Cloudy for over two years. To gain insight into the capacity of end-user services to generate enough value and utility to sustain the whole cloud ecosystem, we have developed a file storage application and tested it with a group of 10 guifi.net users. The experimental results and the experience from the action research confirm the feasibility and potential sustainability of the community cloud as an open commons.Peer ReviewedPostprint (author's final draft

On the Guifi.net community network economics

How costs are distributed among the participants is a key question in the management and viability of shared resources. Although all cost-sharing mechanisms are subjective and thus it is eventually up to the participants to accept one or another, some general criteria seem desirable, such as being budget-balanced and that, in any case, a participant pays more when not cooperating with anyone else. In this paper, we analyse the cost-sharing mechanism that the Guifi.net community network has developed and put in practice to split the transit costs among their more than 20 participants for almost a decade. Our results show that the Guifi.net’s cost-sharing mechanism of the external connectivity, which comprises an equal membership fee for each participant plus a proportional distribution of the remaining costs according to the resource consumption, yields a cost assignment similar to the Shapley value. Our analysis also shows that any alternative to the coalition of all participants entails significant total cost increases and detrimental widespread cost allocation.Peer ReviewedPostprint (author's final draft

Receiver-driven routing for community mesh networks

Community wireless mesh networks are decentralized and cooperative structures with participation rules that define their freedom, openness and neutrality. The operation of these networks require routing algorithms that may impose additional unnecessary technical restrictions in the determination of routes that can restrict the freedom of community users. We propose a receiver-driven discretionary routing mechanism where each receiver (the intended destination of the packet) can freely specify delivery objectives and remain compatible with the collaborative approach of community networks. Each node has a unique identifier and can announce the description of its offer and also the description of its routing policy with preferences to deliver traffic to it. BMX6 provides a 'hash-based profile propagation mechanism' to disseminate descriptions. This receiver-driven routing can be applied to express preferences for desirable nodes and paths, or to restrict traffic to trusted nodes enabling trust and security aware routing. We validate our contributions with a proof of concept implementation of key concepts, as an extension of the BMX6 routing protocol, that confirms its feasibility and scalability.Postprint (author’s final draft

guifi.net, a crowdsourced network infrastructure held in common

The expression “crowdsourced computer networks” refers to a network infrastructure built by citizens and organisations who pool their resources and coordinate their efforts to make these networks happen. “Community networks” are a subset of crowdsourced networks that are structured to be open, free, and neutral. In these communities the infrastructure is established by the participants and is managed as a common resource. Many crowdsourcing experiences have flourished in community networks. This paper discusses the case of guifi.net, a success case of a community network daily used by thousands of participants, focusing on its principles and the crowdsourcing processes and tools developed within the community, and the role they play in the ecosystem that is guifi.net; the current status of its implementation; its measurable local impact; and the lessons learned in more than a decade.Peer ReviewedPostprint (author's final draft

A look at energy efficient system opportunities with community network clouds

Community networking is an emerging model of a shared communication infrastructure in which communities of citizens build and own open networks. Community networks offer successfully IP-based networking to the user. In addition, some hosts are connected to the network nodes in order to provide network management and end user services. Recently, clouds have been proposed for community networks. Some research projects such as Clommunity have started deploying computational infrastructure to enable cloud computing within community networks. In this paper we propose different options for such community clouds to contribute to energy efficient systems, in particular regarding cloud-based services and in relation to Smart Grid. Further discussion and interaction with the research initiatives on energy efficient systems should identify the most promising approach and outline possible ways for implementation.Peer ReviewedPostprint (published version

On edge microclouds to provide local container-based services

Edge computing has been proposed to enable more user-centric cloud-based services. Nowadays, edge computing is operational in industrial and consumer-oriented scenarios. An important limitation of today's solutions, however, is that the used hardware and software platforms are proprietary and closed, and cannot easily be leveraged to perform other services beyond the specific business case. For instance, the interaction among different edge platforms or service extension by third parties is in general not supported. As a consequence, the opportunity for local stakeholders to provide innovative tailored edge service with these platforms face important barriers. In this paper we present edge microclouds deployed on local servers and an implementation using containers for service provision. We show how the adopted container approach facilitates the users to create and share services at the network edge. With the presented approach, third parties can deploy more tailored and customized services at the network edge, enabling to better fulfill specific local needs and constraints. It is also an opportunity for building cloud-based service provision with a pool of local resource-constraint edge devices.Peer ReviewedPostprint (author's final draft

Advances in wireless community networks with the community-lab testbed

Beyond traditional telecom providers, citizens and organizations pool their own resources and coordinate in order to build local network infrastructures to address the digital divide in many parts of the world. These crowdsourced network infrastructures can be self-organized and shared by a community for the collective benefit of its members. Several of these networks have developed open, free, and neutral agreements, and are governed as a common-pool resource: community networks. These are built using a variety of commodity wireless hardware (e.g., Wi-Fi long-range point-to-point links, Wi-Fi and GSM access points, and mesh networks), sometimes optical fiber links, heterogeneous nodes, routing protocols, and applications. A group of researchers, developers, and community networks developed the Community-Lab testbed, and for the last five years have worked together to overcome obstacles, improve the technologies, tools, and operational models being used, as well as model best practices for more effective and sustainable community networks. This article presents the challenges for experimentation, the testbeds built, results, lessons learned, and the impact of that work to place wireless community networks as one sustainable way toward an Internet accessible to all.Peer ReviewedPostprint (author's final draft

A technological overview of the guifi.net community network

This work presents a technological analysis of guifi.net, a free, neutral, and open-access community network. Guifi.net consists of more than 27,000 operational nodes, which makes it the world’s largest community network in terms of the number of nodes and coverage area. This paper describes the characteristics of the network, the link level topology, its growth over a decade, and its resilience in terms of availability and reachability of network nodes. Our study is based on open data published by guifi.net regarding its nodes and wireless links, monitoring information, community database, and its web portal. The data includes historical information that covers the lifetime of the network. The scale and diversity of the network requires a separate analysis of the subsets of the entire dataset by area or by separating the core from the leaf nodes. This shows some degree of diversity in local characteristics caused by several demographic, geographic, technological, and network design factors. We focus on the following aspects: technological network diversity, topology characteristics, evolution of the network over time, analysis of robustness, and its effect on networking service availability. In addition, we analyse how the community, the technology used, the geographical region where the network is deployed, and its self-organised structure shape the network properties and determine its strengths and weaknesses. The study demonstrates that the guifi.net community network is diverse in technological choices for hardware, link protocols, and channels and uses a combination of routing protocols yet provides a common private IP network. The graph topology follows a power- law distribution for links in regions up to a few thousand Km 2 , limited to the scope of wireless links. Network growth has two aspects: a geographic growth of the network core using long distance links with wireless or fibre, and the local growth in density with leaf low-cost leaf nodes. The resilience of the network derived from the nodes’ uptime and the structure of the graph varies across different regions with more fragile leafs than core nodes and diverse degrees of graph resilience to random failures or coordinated attacks, such as natural causes, depending on the network planning, structure, and maturity. The guifi.net community network results from a loosely coupled and decentralised organic growth that exhibits large local differences, diverse growth, and maturity under a common community license and social network.Peer ReviewedPostprint (author's final draft